Prior to the background of the invention being set forth, it may be helpful to set forth definitions of certain terms that will be used hereinafter.
The term ‘structured light’ as used herein is defined as the process of projecting a pattern of light on to a scene. The pattern is projected by a ‘transmitter’ which includes an illuminator which creates a light beam and optics which converts the light beam into the pattern. The way that this pattern deforms when striking surfaces allows vision systems to calculate the depth and surface information of the scene and the objects in the scene. The pattern is being captured and analyzed by a so-called ‘receiver’ which includes a sensor (or a capturing device) and a computer processor for analyzing the captured reflected pattern.
Invisible structured light may be used without interfering with other computer vision tasks for which the projected pattern will be confusing, by typically using infra-red light IR.
Structured light has many applications. One such application can be the identifying and the tracking of real objects in a scene as explained in further details in US Patent Publication No. US 2012/0194561 titled “Remote control of computer devices” and in WIPO Publication No. WO 2015/059705 titled: “Three dimensional depth mapping using dynamic structured light” both of which are incorporated herein by reference in their entirety.
The term ‘depth map’ as used herein is defined as an image that contains information relating to the distance of the surfaces of scene or objects in the scene from a viewpoint. The computer processor of the receiver generates the depth map using prior knowledge of the light pattern. Specifically, the analysis of the reflections coming from the scene is based on a triangulation process in which the location of every point of the pattern is derived based on the relative location and orientation of the transmitter and the sensor.
One of challenges of using structured light 3D sensing is how to overcome ambient light which contributes to the noise level of the sensor. The problem with ambient light is that when it is at a high level, the signal to ambient ratio becomes low and leads to poor performances. As one example, the sensor noise is typically related to the overall illumination. A strong ambient light will therefore increase the system noise and reduce the signal to noise ratio.
As another example, the sensor's spatial non-uniformity is typically also related to the illumination level. Ambient light will therefore increase the non-uniformity while not contributing to the signal level. The ambient light (or background illumination) eventually affects the range and field of view (FOV)—the area covered by the pattern—per unit power of the system. This is because it is necessary to keep the structured light pattern at a certain level of brightness above the background.
Currently, all familiar structured light techniques use a staring camera as a sensor for the receiver. As such, the camera allows all the ambient light to enter throughout the full integration time which is related to the frame time of the camera (the time a single frame is being captured). In a staring camera system, the only way to reduce the ambient light coming into the sensor is to use a shorter exposure time. This, however, requires operating the laser (the illuminator) in high power short pulses which increase complexity and reduces laser reliability. Using short pulses may also be limited by eye safety issues.